BlueOnyx 5211R Development Diary

Further progress report on the BlueOnyx 5211R development

The development of BlueOnyx 5211R on (or for) AlmaLinux 9 / RockyLinux 9 / RHEL9 is progressing nicely, as reported earlier. Since then many important milestones have been reached and it is time for another progress report.

But before I do that, please note that developing BlueOnyx 5211R is a monumental task that requires a lot of effort and dedication. If you use BlueOnyx and would like to use BlueOnyx 5211R in the future? Then please consider supporting the development with a donation.

Donations:

BlueOnyx is available free of charge. For all purposes, may they be personal, commercial, educational or whatever else you might want to use it for. However: Keeping BlueOnyx updated, adding new features to it and releasing updated ISO images is a lot of work. If you like BlueOnyx, then please consider donating something to the project. Thank you and enjoy using BlueOnyx!

State of affairs for BlueOnyx 5211R:

AdmServ and PHP:

AdmServ now uses PHP-8.1.10 via PHP-FPM and the PHP-8.1.10 is installed under /home/solarspeed/admserv-php/ and is therefore independent of the OS provided PHP version. This version of PHP-8.1 ships with the IonCube Loader for PHP-8.1.

During the lifecycle of BlueOnyx 5211R this PHP will be constantly updated by us to the latest PHP version.

PHP Framework of the GUI:

The GUI uses the latest CodeIgniter v4.2.5, which we will also keep current throughout the lifecycle of BlueOnyx 5211R as far as required to maintain current with the used PHP version.

State of dependencies and software packages:

The following functionality from BlueOnyx 5210R will be absent on BlueOnyx 5211R:

• Mailman
• Z-Push Active Sync
• Web Based File manager

Mailman isn't available for EL9 and my attempts to build it from scratch haven't been met with success. So I set this aside for a later time.

Z-Push Active Sync and the Net2FTP integration we had in the BlueOnyx 5210R GUI are now dead projects and won't work on modern PHP such as PHP-8.0 or PHP-8.1. So these have been dropped.

A web based filemanager via the GUI will most likely be added later on again, as I have some ideas how we can fill that gap.

Other than that? I do have 90% of all odds and sods sorted by now, so via "cceclient" almost the full spectrum of functionality is available at this time: Creating, managing and deleting Vsites, Users and all other functions one would expect from a working BlueOnyx. The remaining 10% will be addressed during the completion of the work on the GUI itself.

GUI for BlueOnyx 5211R:

As mentioned earlier: CodeIgniter 4.2.5 is radically different from any CodeIgniter version that BlueOnyx used before. This means that there is no way to do a straight upgrade and continue with existing code. Basically everything has do be done at least *slightly* different.

So I started from scratch and solved a metric f-ton of systemic issues. Beginning with writing my own dynamic URL-to-Class-loading routing mechanism, modified the CSRF mechanism in CodeIgniter 4 to match our architecture, stomped hard onto their input verification and filtering system to integrate our own and did many other architectural adjustments to keep a resemblance of our previous code structure and especially the core of our GUI functionality alive.

Then I started small and eventually got the BlueOnyx login page up and working, the /swupdate/news page, all Active Monitor pages, all base-backupcontrol and all base-console pages.

Initial progress was slow, as each new page I ported had many dependencies and used libraries or functions that needed adapting to PHP-8.1 and/or the new internals of CodeIgniter. By now the process of porting a single old GUI page to a new one is down to 45-60 minutes (including testing), 90 minutes if it's a really complex page with lots of options.

While working on the porting of the old GUI pages to the new CodeIgniter I also made full use of the extended debugging and code runtime profiling that is now possible if CodeIgniter runs in development mode. This allowed me to identify some horrific performance bottlenecks, which I then directly addressed.

Which brings me to the next point.

GUI PERFORMANCE:

AdmServ is now using HTTP/2 out of the box, which delivers content in a parallelized fashion. Means: That is already faster out of the box.

Usage of PHP-FPM and PHP-8.1 also speeds things up considerably compared to PHP-7.2 and DSO like one BlueOnyx 5211R.

The new CodeIgniter 4 and the elimination of some bottlenecks and redundancies in the new GUI also speeds things up considerably.

BUT: The biggest speed boost was yet to come and that's a little something I hacked together on the sides: CCE-Caching.

When you load a single GUI page, this page also loads two dynamic GUI pages as dependencies for localization of error messages during input validation as well as the error messages provided by jQuery.

That runs three instances of the GUI-code for a single (visible) GUI page. Each of these wants (or needs) to poll CCE for certain CODB objects:

• "System" Object
• "User" Object of the logged in GUI user
• "Capabilities" Object and its 40 NameSpaces for your access rights
• "Active Monitor" Object and all its NameSpaces for system faults

Plus whatever CODB Objects the GUI pages needs to display the *actual* information you want/need to perform the desired administrative function(s).

Usually these objects are fetched via $cce->getObject("Class"). Which runs a "FIND Class" against CODB and finally a "GET <OIDs>". In itself that are already three CCE transactions for a single "give me data!" task.

Means: Even if we do nothing else but load a GUI page, we do have 12 CCEd transactions, of which 2/3rds are repetitive.

Welcome to CODB Caching \o/

We do have single instance CODB Objects, which rarely change during the lifetime of a GUI session. And coincidentally and conveniently these are also exactly those that we use the most. Even if they change? Then we can deal with that easily and can STILL cache them.

So that's what I did.

The GUI now uses the PHP session data as storage for those CODB objects that are mostly static and are most used during your GUI session. If the data is already in the session-data Cache, CCEd will not be asked, but instead the data will be fetched from the cache.

Additionally and index of CODB OIDs will be kept in the session-data cache, so that we can skip redundant "FIND" transactions against CODB, because we already know what the Object ID of the desired Class is.

Say you login and land on the start page of the GUI: /swupdate/news

During the initial login (before I implemented the Cache) around 56 CCEd transactions had to happen. If you then simply reloaded the /swupdate/news while remaining logged in? Another 20 CCEd transactions had to happen.

Now with the Cache around 16 CCEd transactions happen during initial login. From seeing the login page to landing on /swupdate/news, as anything important and cache-able will be fetched only once and then be served out of the Cache.

Reloading the page past the login? Only 8 CCEd unavoidable transactions are required (3x authentication, 3x ending the connection to CCE, getting the data we want to display), as the rest is served straight out of the Cache.

How much faster is this?

On my test hardware we save around 20-25 milliseconds per FIND/GET transactions. This adds up nicely.

Direct comparison:

Login into 5210R:

Load time /login page:            3.5 seconds
Login and loading /news/swupdate: 6.8 seconds
Reloading /news/swupdate:         6.5 seconds

Login into 5211R:

Load time /login page:            2.3 seconds
Login and loading /news/swupdate: 4.9 seconds
Reloading /news/swupdate:         3.5 seconds

Remaining work prior to release of the BETA:

GUI pages:

I still have plenty of GUI pages to port and test. At current speeds I expect I need another three weeks for those.

Packaging:

CodeIgniter 4 has a different directory structure than CodeIgniter 3 and also needs slightly different Routes.php and Controllers/*.php files. Keep thing simple for eventually backporting the 5211R GUI to 5210R (which I might!) I decided to install all GUI related files into slightly different folders.

Subsequently I need to modify the SVN file structure for BlueOnyx modules and the packaging script slightly to deal with this. That will take a few days before packaging and automated RPM building is sorted.

YUM Repository:

It's sorted and up. Most OS related bits and pieces are published. The new GUI RPMs will be published when the packaging is sorted.

ISO Building:

Not yet started, but I expect few surprises. I should be able to adapt the 5210R ISO building procedure nicely to handle EL9 ISO's as well. 

BlueOnyx 5211R release date:

I don't like to make promises on that. However, the coast is pretty clear from here and there are no insurmountable mountains to climb anymore. The rest is just nitty-gritty digging in the heels and cranking code and procedures out. If nothing unforeseeable happens, I think a November release of BlueOnyx 5211R is quite doable.